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Multivariate sequential analysis with 
linear boundaries 
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, The protocol for most sequential experiments dictate that data are collected until a 

' random walk {Xn}n>o crosses a curved boundary. For design and analysis of these 

experiments there has been great interest in the joint distribution of the sample 
size r and X^.; or related marginal distributions. Using nonlinear renewal theory or 
other techniques, it is often possible to approximate distributions for experiments 
^ ' with curved boundaries by distributions when the boundary is linear. The results 

OO . presented here all concern the linear case with sample size 

' T ~ Ta = inf{n : Xn > a}. 



University of Michigan 

Abstract: Let {Sn = {Xn,Wn)}n>0 t>e a random walk with Xn G M and 
Wn G M"'. Let T = Ta = inf{n ; X„ > a}. The main results presented are 
two term asymptotic expansions for the joint distribution of St and t and 
the marginal distribution of h{ST /a,, r/a) in the limit a — > oo. These results 
are used to study the distribution of 4-statistics in sequential experiments 
with sample size r, and to remove bias from confidence intervals based on 
Anscombe's theorem. 



1. Introduction 



As a ^ oo, the sample size r is asymptotically normal (after suitable centering and 
scaling); the excess over the boundary, S-r — a, has a limiting distribution; and these 
variables are asymptotically independent. For this result, its extension to curved 



asymptotic expansions (with curved boundaries) are given in i, mill, [3 Hi, H 



o 

I boundaries, and applications to sequential testing and estimation, see [2j] or [17[. 

' In applications, a may not be very large, and to obtain an adequate approxima- 

^ I tion it is often necessary to refine the basic limit theory. In univariate situations 

> 



, . Mm- 

' In this paper, the primary concern will be with distribution theory for multi- 

^ , variate experiments in which several variables are recorded for each experimental 

unit. In practice, the stopping time will often be driven by the values for some pri- 
mary variable, but parameters associated with distributions of the other variables 
will also be of interest. In a clinical setting, the primary variable might be some 
measure of the efficacy of treatment, with other variables related to potential side 
effects. Since the variables not directly related to stopping may be correlated with 
the primary stopping variable, a statistical analysis that ignores effects of optional 
stopping may be biased, even if the variables involved are not directly related to 



the stopping rule. Whitehead [24I notes this possibility and provides adjustments 



to reduce the bias of maximum likelihood estimators in these situations. 
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Most of the expansions cited above for univariate data are based on arguments 
that do not extend easily to higher dimensions. A notable exception would be an 
approach introduced by Woodroofe [26[. This approach is based on the likelihood 
function in a parametric context, making it particularly appropriate for sequen- 
tial or adaptive experiments since likelihood functions are not effected by optional 
stopping. With this approach, coverage probabilities for approximate confidence 
intervals are viewed as functions of the unknown parameter. Expansions for these 
coverage probabilities do not hold in the conventional pointwise sense. Instead, the 
expansions hold if the functions are smoothed by integration against some weight 
function. Woodroofe calls these results "very weak expansions." Very weak expan- 
sions have now been used in a variety of situations to set confidence intervals for 
secondary parameters— see d [H, IM [HI, IH, [H, 113, El . 



One final approach to interval estimation deserves mention. In one dimension, 
Siegmund suggests ordering points in the stopping region and inverting a fam- 
ily of tests. Probability calculations in this approach are based on large devia- 
tion approximations, which gives reasonable accuracy in many standard designs — 
especially repeated significance tests. In higher dimensions, it is less evident how 
points should be ordered in the stopping region. However, Siegmund in [l^ success- 
fully extends his method to interval estimation following a repeated significance test 
of several normal means. Unfortunately, his argument uses the spherical symmetry 
of the normal distribution and may not be widely applicable. 

In this paper, expansions are derived from related expansions for multivariate 
renewal measures given in |12| . In contrast with very weak expansions, a para- 
metric model and likelihood function are not necessary, and the expansions hold 
in the conventional sense — no smoothing is necessary. Unfortunately, the algebra 
necessary with this approach seems more difficult. Expansions are only obtained 
with a leading correction term, and, as mentioned above, the stopping time r must 
have a linear boundary. The main results give expansions for joint distributions 
of partial sums and the stopping time. Using these results, refined approximations 
for the distributions of t-statistics are derived and used to set confidence intervals 
for the mean of a secondary variables. The refined distribution theory reduces tail 
probability bias present in confidence intervals set using the normal approximation 
based on Ancsombe's theorem. 

Extensions of the results in this paper to curved boundaries seem challenging 
but may be possible. In the univariate case, considered in [llj, under conditions 
sufficient for local limit theorems the curvature of the boundary only has an intrinsic 
effect on the second correction term — the chance of crossing a curved boundary on 
step n agrees with the chance of crossing an appropriate linear boundary at step 
n to o(l/a). But curvature does have some effect on the first correction term since 
the appropriate linear boundary must vary with n. Using this, it may be possible 
to guess how the results here could be modified for curved boundaries, although 
the best approach for a rigorous argument is not clear. 

Let {{Xn, Wn)}n>o bc a random walk with X„ e R and VK„ e M™. Theorem [TTT] 
provides an expansions for the joint distribution of Xr and Wr and the joint dis- 
tribution of Xr, Wr and r. To state this theorem, a fair amount of notation is 
necessary. Let (f> denote the normal density with mean zero and identity covariance. 
For any function /, the oscillation function w/ is given by 



uif{x; e) = sup{|/(x) - f{y)\ : \x - y\ < e}. 
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The random vector {X, W) — {Xi, Wi) satisfies Cramer's condition if 



limsup |£;e*(«i-^+«^-'^)| < 1. 

(«l,?2)^00 

For joint expansions with t, let W* = (W^„,n) and take W* — . Then W* = 
(WV,t). Let V = EX, 7 = EW/v, 7* = EW*/u, E = Cov{W - 7X), = 
Cov(W^* - 7*X), Z„ = S-i/2(W^„ _ 7X„), = J:-'/\w* - rXn), Z = Zi and 
Z* = Zl. The first ladder time is T = tq. Let X ^ Xt,W = Wt, W* = W^, 
Z = ZtI^/ET and Z* = Z'^j^KT. Define 



and 



Po(a;) 



_ P{X > x) 

^Jet ' 

E[Z;X> x] 
vy/ET 

E[Z*-X > x] 

vVet 



for a; > 0, with po, pi and p\ identically zero on (— oo,0]. The densities for the 
approximate distributions in Theorem 1 1 . 1 1 are given by 



and 

dQ* 



dX* 



dQ 



S|(a/z/)'»/2 



|po(a;) + y/vJa\H{q)pQ{x) + q ■ pi{x)] | 



^(a/^)(™+ 



Jy^{poix) + \^Wa[Ji*iq*)po{x) +q* ■ plix)]^ 



where A is Lebesgue measure on M™+^, A* is the product of A with counting measure 
on Z, 

q = Yi^^^^{w — jx — ^a)\p\7fa, 



{m + 2-\q\^)EXq- Z EXq-Z 



q 



m.q)^\E{q-Zf~]^EZ\-Z 



and 



2v 



H*((7*) = -E{q* ■ Z*f - -EZ*^q* ■ Z* 
6 2 



i^VET 



+ 



(m + 3 - \q*\'^)EXq* ■ Z* EXq* ■ Z* 



2v 



lyVET 



Theorem 1.1. Suppose {X,W) satisfies Cramer's condition, v > 0, E\X\'^^^^'>/^ < 
00 and E\Z\^^^ < 00, where S £ (0, 1). Then for some r/ > 0, 

Ef{Xr~a,Wr)^ I fdQ + 0{l) f ujf{--e-^'')dQ 



+ o{a(-i-*')/2(loga)'"/2} 
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as a oo, uniformly for measurable nonnegative f bounded above by 1. If the 
moment condition for X is strengthened to E\X\^^^ < oo, then for some rj > 0, 

Ef{Xr~a,W:)= J fdQ*+0{l) J ujf{-;e-^'')dQ* 
+ o{a(-i-^)/2(loga)('"+i)/2} 

as a —> oo, uniformly for measurable nonnegative f bounded above by 1. 

With / an indicator function, this theorem provides approximations for prob- 
abihties. Unfortunately, access to the wealth of information available in principle 
from this result may be rather difficult. The next result uses Theorem 11.11 to ap- 
proximate the marginal distribution of h{Xr/a, Wr/a, r/a) for smooth functions h. 
Special cases of interest include linear functions, averages, normalized partial sums 
and t-statistics. 

To state this result, let Yn denote the first coordinate of Wn, so Wn = {Yn, Vn) 
where Vn € K™"!. Also, let = (y„,n), so W* = (Y^.V*). Finally, let 5; = 
(5'„, n) — {Xn,Yn, Vn, n) . By the strong law of large numbers, Xr/a 1, Yt/o — > 71 
and V*/a ^ 72 as a — > 00, where 71 = EYjv and 72 = EV* /v. The expansion 
naturally involves Taylor expansion of h about sq = (1,71,72). Let V«* and V^* 
denote the gradient vector and Hessian matrix with respect to v* . The regularity 
assumptions for h are as follows: 

1. The mixed third derivatives of h are continuous in some neighborhood A'o of 
So (this assumption could be relaxed slightly). 

2. At So, h = dh/dx = 0, V„./i = and dh/dy = 1 

Since invertible affine transformations of (A", W) preserve the moment conditions 
and Cramer's condition, this last assumption is less restrictive than it may appear. 
The only cases not covered are those where the first order Taylor expansion of 
h{S*/a) is linearly independent of Wr, and hence depends only on X^ and t. 
Define 

2 9^2 



^0 = 0—^(^0), 



A^^Wl,h{so), 
and let hi be the linear function given by 

d 

hi{v*)^v* • V^.— /i(so). 

dy 

The quadratic Taylor approximation for h about Sq is 

(y - 71) + hfiiy - 71)^ + (y - li)hi{v* - 72) 

+ {v* - 72) • A{v* - 72) + hz{x - 1)^ + {x- l)hi{y - 71, w* - 72), 

where /13 is a constant and ^14 is a bi-linear function — their specification is not 
important. Partition as 

_ /Sii E12 

\S21 S22, 
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where T.22 = Cov{V* - 72X) is m by m and En = Var(r - jiX) = a^. The 
approximate distribution function in Theorem 11.21 is 



6 2v 



(c^-l)-^ 
^ ' 2v(jET 



-Zj12A2j21 



va va-^ 
where £ = c^/v/ a. 

Theorem 1.2. Suppose Q < 5 < 1, E\X\^+^ < 00, E\W\^+^ < 00 and {X,W) 
satisfies Cramer's condition. Also, if 5 > V2 ~ 1, assume _B|X|2/(i~*) < 00. Let 

S = y/ah{Xr/a, Wr/a, r/a), 
where h satisfies the conditions stated above. Then 

P(S < c) = Faic) + o{a(-i-^)/2(loga)(™+i)/2} 
as a —> 00, uniformly for c G M. 



The rest of the paper is organized as follows. In the next section, Theorem 1 1.2 1 is 
specialized to t-statistics and used to set confidence intervals for EY. Results are 
reported for a simulation study comparing the coverage probabilities of these con- 
fidence intervals with confidence intervals set using Anscombe's theorem. Section |3] 
contains an approximation for the joint distribution of Xj- and Wr in the positive 
case where X > 0. In Section 21 ladder variables are introduced and used to prove 
Theorem 11.11 from results in Section [3] In Section [SJ various marginal distributions 
are approximated and Theorem 11.21 is proved. 

2. Confidence intervals and t-statistics 

The main concern of this section is setting confidence intervals for fi = EY after a 
sequential experiment with sample size r. Let e„ = Yn — Yn-i, so y„ — X^ILi ^i- 
The ^-statistic for /i is 

Yr-fl 



T 



where Yr =Yr/T and 



a^ 



T 

By Anscombe's theorem [l|, 

(1) T=>X{Q,\) 

as a — > 00. Using this, the coverage probability of the confidence interval 

(LCLo,UCLo) = (F, ± 

approaches 1 — 2a as a ^ 00, where = $^^(1 — a). 

The normal approximation ([1]) can be improved using Theorem 11.21 It is most 
convenient to work initially with the modified statistic 
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where CTq = (r — 1)(7^/t. If Vn ~ J2^=i then 



2 

r t' 



.2 



and 

To — \fah{YT I a,Vr I a,T I a), 

where 

h{y,v,t) — — ^ ^ :■ 

Corollary 2.1. If {X,Y,Y'^) satisfies Cramer's condition, v > 0, £'|Xp+'5)/2 < 
oo, £;|X|2/(i-'5) < oo and E\Y\^+^^ < oo, f/ien 

P(To < c) « + 0(0)^^ {0(1 + 2c^) - 1^ 

uniformly in c as a oo, where = Var(y), /is = E{Y — ^)^, T,xy = Cov{X, Y). 
Here and later in this section, " will mean that the two quantities differ by 
o{a(-i-^)/2(loga)3/2}. 

Proof. After recentering Y and rescaling X, Y and a, there is no harm assuming 
/i — 0, <T = 1 and v = I. Then c ^ c, ji — and 72 = (1, 1). Differentiation gives 
ho — 0, h2 — and hi{v,t) — — w/2, so 



Fa{c) = $(C) + 



0(c) 



„2 



(c^ - 1) + y i^(i^^ - XY) 



and the coroUary follows from Theorem II. 2 1 □ 

The next lemma is a generalization of Slutsky's theorem to asymptotic expan- 
sions. Using it, the approximation in this corollary is also an approximation for the 
distribution of T. By definition, say that Ca — > in probability at rate o{5a) if for 
every e > 0, 

PQCal > e) = 0{Sa) 

as a — > 00. 

Lemma 2.2. Suppose the distributions of a family of variables {^a} have an asymp- 
totic expansion of the form 

P{Ca < C) = Ga{c) + o{ea) 

as a 00, uniformly in c, where the Ga satisfy 

lim sup sup sup{ Ga(x -\- y) — Ga{x)} /y < 00. 

a — ^00 X y>0 

Then if Ca in probability at rate o{ea), 

P{^a+€a(a<c)^Ga{c)+0{ea) 

as a —f 00, uniformly in c. 
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Proof. Let 



K = sup supsup{G'q(x + y) - Ga{x)}/y. 

a>ao X y>0 



Then if < c» if oq is large enough. For any e > 0, for a > ag, 

P{^a + eaCa < c) < P{^a < C + + P{Ca > e) 

= Ga{c + €ea) +0{ea) 

<Ga{c)+Keea + o{ea) 
as a — !■ cx), uniformly in c. Since e is arbitrary, 

^'(^a + eaCa < c) < Ga(c) + o(e,) 
as a — > cxD, uniformly in c. The reverse inequality follows in a similar fashion. □ 



Using Lemma I2.2[ it is easy to check that T and Tg have the same asymptotic 
expansion (to this order). For setting confidence intervals, it is convenient (following 
Hall [13]) to write the expansion in the form 



P T <c 



M3 

6a3 



(1 + 2c2) 



-•XY 



2vcj 



as a — + cxD. This expansion may not hold uniformly in c as a — )■ oo, but will hold 
uniformly for c in any compact set. It is now natural to replace the parameters v, 
(T, /i3 and SxF by estimates v, <t, fi^ and £xy- By the same calculations used to 
prove Lemma |2.2[ if 

a'^/2(ioga)3/2{|p -y\ + \a-a\ + 1^3 - Msl + \txY - Sjfy|} 
in probability at rate o{a'^^^*'/^(loga)'^/^}, then 



P\T<c-\l- 
a 



OCT'' 2,V<J 



as a "> oo, uniformly for c in any compact set. With c = ±Zq,, the coverage proba- 
bility of the confidence interval 



(LCLi,UCLi) Y, 



a Iv 
7^"^ a 



M3 

6ct3 



(1 + 2^ 



^XY 
2i>CT 



is ~ 1 — 2q: as a — > OO. 

In parametric estimation following a sequential test, Whitehead [l^] suggests 
centering confidence intervals around the maximum likelihood estimator less an es- 
timate of its bias. In the normal case where Yr is the maximum likelihood estimator 
for /i, the interval suggested here is different: the center to the relevant order is Yr 
less half its bias. 

Corollarv l2.1l can be used to study the performance of the interval (LCLq, UCLq). 
By the corollary. 



P{H > UCLo) « a - 
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and 

< LCLo) « a - + 2zl) - ^ j , 

so 

P(LCLo < M < UCLo) w 1 - 2a. 

To this order of analysis, the overall coverage probability equals the desired value, 
1 — 2a, but this confidence interval is biased — the two error probabilities, P(/i > 
UCLq) and < LCLq) do not agree. The modified interval (LCLi,UCLi) has 
no bias to this order of analysis: 

P{p > UCLi) ^a^ P{fi< LCLi). 

Although the modified confidence interval has less bias and comparable coverage 
as a — > oo, there is the usual concern that these properties may fail for moder- 
ate values of a. Extra variation is introduced when the parameters v, a, /is and 
Exv are estimated, and this may degrade the overall coverage of the interval. The 
performance of several confidence intervals are studied in the simulation study re- 
ported in Table 1. In this study, {X, Y) follow a bivariate normal distribution with 
Yar{X) = Var(y) = 1, so Exy equals the correlation coefficient p. Two versions of 
the modified confidence interval are considered. For both, u and T,xy are estimated 
as 

i, = X^= Xr/r 

and 

1 ^ 

txY = - y{X, - X,_i)e, - XrYr. 

T ^ — ' 

i=l 

For the interval (LCL^^'', UCL^^''), fl^ — 0. This would be appropriate in practice 
if the researcher were fairly certain the centered marginal distribution of Y was 
symmetric. For the interval (LCLj^ ,l]CL\ ), is estimated by 

In the simulation, a was fixed at 5%. There were 10,000 replications. 

Examining the simulation results a few tentative conclusions seem in order. There 
is almost no difference between the performance of the two versions of the modified 
confidence intervals. The original confidence interval has better coverage, but only 
slightly — less than one percent. The original confidence interval can be quite biased 
with more that a 2 to 1 ratio between the error probabilities in some cases. The 
modified intervals were always less biased. Finally, it is worth noting that in all cases 
the overall coverage is a bit less less than 90%. Higher order corrections would be 
useful. 

3. Positive case 

The notation in this section parallels the notation in the introduction, but is some- 
what divergent. Let {{Xn,Wn)}n>o be a random walk with X ~ Xi > 0, and 
W = Wi € R™. The stopping time is still t = Ta = inf{n : Xn > a}. The main 
result of this section is an asymptotic expansion for the joint distribution of X^- and 
Wr- When this result is used proving Theorem ll.il {X, W) will be either {X, W) or 
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Table 1 

Coverage Probabilities and Error Probabilities 



a =10 a = 25 



1/ = 0.5 u = 0.25 u = 0.5 v = 0.25 



■ OA p = 0.8 p = OA = 0.8 = 0.4 p = 0.8 p = 0.4 



(LCLo,UCLo) 
Piu,> UCLo) 
p\ix < LCLo) 

lclW,uclW) 

P(/i < LCL<^') 
(LCLf ,UCL^^') 

p{^l> UCL^^^) 



0.884 


0.881 


0.892 


0.894 


0.890 


0.887 


0.890 


0.893 


0.047 


0.035 


0.045 


0.030 


0.051 


0.047 


0.050 


0.042 


0.069 


0.083 


0.064 


0.075 


0.059 


0.066 


0.061 


0.065 


0.880 


0.876 


0.892 


0.879 


0.890 


0.884 


0.883 


0.885 


0.057 


0.055 


0.057 


0.062 


0.057 


0.061 


0.064 


0.062 


0.063 


0.069 


0.051 


0.060 


0.054 


0.055 


0.054 


0.053 


0.874 


0.873 


0.892 


0.876 


0.888 


0.885 


0.882 


0.885 


0.058 


0.056 


0.059 


0.063 


0.058 


0.060 


0.063 


0.061 


0.068 


0.071 


0.050 


0.060 


0.054 


0.056 


0.055 


0.054 



(X, W*). Since the last coordinate of W* is T, two different smoothness conditions 
will be used. In the "continuous" case, {X, W) will satisfy Cramer's condition. In the 
"mixed" case, with W = {Y,T) say, T will be arithmetic on Z, i.e., P{T e Z) = 1 
but P{T e B) < 1 for B any proper subgroup of Z, and {X, Y) will be strongly 
nonlattice with T, i.e., 

liminf inf \l - Ee'^^^+'^^-^+'^''^\ > 0. 

In the continuous case, A will denote Lebesgue measure on M'"+^ , and in the mixed 
case A will denote the product of Lebesgue measure on with counting measure 
on Z. Also, define A by 

X{B) = X{{{x,w) : (a;,E-i/2(w - 7a;) e B}) 

for Borel sets B. 

Let 1^ = EX, 7 = EW/i^ and S = Cov{W - jX). Assume < S < 00 and 
take Z„ = E-i/2(W„ - J^n) and Z = Z^. Let {X^, Wn) ~ F„ and (X„, Z„) ~ 
with F = Fi and F = Fi. Then R = J2n>o ^ — Sn>o ^n are the renewal 

measures for the random walks {{Xn,Wn)}n>o and {(X„,Z„)}„>o respectively. 

Theorem 3.1. Suppose < S < 1, £;|X|(3+'5)/2 < 00, E\Zf+^ < 00 and T, is 
positive definite. Also, assume that either {X,W) satisfies Cramer's condition, or 
W = {Y, T) with T arithmetic on Z and {X, Y) strongly nonlattice with T. Then 
for some r] > 0, 

Ef{Xr-a,Wr)= J fdQ + 0{l) J u;/(-;e-'"^)dQ + o{a(-i-*)/2(i^g^yn/2| 

as a ^ 00, uniformly for measurable nonnegative functions f bounded above by 1, 
where for x>Q, 

^^""'""^ ^ ^(!/i)m/2 {p°(^) + V^[H(g)po(a;) - q ■ p,{x)\ }, 
q = — 7a; — ^a)\/v/a, 



Sequential analysis 67 

H(g) = lEiq ■ Zf \eZ\ ■ Z + ^^^-^EXq ■ Z, 
6 2 2v 

Poix)=P{X>x)/iy, 

and 

pi{x) = E[Z;X> x\/v. 

The expectation of interest here can be expressed as an integral against the 
renewal measure R. Since X > Q, the event t = n + 1 is the same as Xn < a and 
Xn+i > a. Hence 

oo 

Ef{Xr - a, Zr) - ^ - a, Z„+i);t = n + l] 

oo ^ ^ 

(2) f{x + xo-a,z + zo)dFn{x,z)dF{xo,zo) 

iCQ<x<.a 

f{x + xo — a, z + zo) <ii?(^, ^) dF{xQ,zo). 



n—0 , ^ 



a-~XQ<x<a 



Expansions for multivariate renewal measures are studied in |12| . To proceed we 
will use the following result which follows almost immediately from Theorem 3 of 

Theorem 3.2. Under the assumptions of Theorem \3.1[ for some rj > 0, 



f{x,z)dR{x,z)^ J [fix,z) + Oil)iUjix,z;e-'n]dRix,z) 

a<2;<a+A a<a;<a+A 

+ o{(l + A)a(-i-^)/2(logar/2} 
as a — > oo, uniformly for A > and nonnegative measurable f bounded by 1, where 

^^\x, z) — r(x,z) — ; < 1 + \/i'/xtl(z\/iy/x] \ . 



Proof of Theorem VJ.l\ The integral in ^ over x > a/2 or \z\ > ^/a as a ^ oo is 
o{a(-^-'^)/2}. To see this, note first that 

dR{x, z) ^ R{{a - xo,a] x R") 

a—XQ<x<a 

which is bounded by a multiple of 1 + xg, uniformly in a and xq > hy the renewal 
theorem in one dimension. Then 

(1 + xo)dF{xo, zo) =E[1 + X;X> a/2] 

, , xo>a/2 

<E[{1+ ^ > ^/2] (1 + a/2)(-i-*)/2 
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and 



(4) 



J {l + xo)dFixo,zo)^E[l + X;\Z\>^] 



\zo\>V^ 



<E[il+X)\Z\'+';\Z\>V^]c 



{-l-S)/2 



as a oo. We can now use the approximation of Theorem 13.21 in This gives 
Ef{Xr - a, Zr) 

[f{x + xq - a, z + zq) + 0{l)u!f{x + xo - a,z + zq; e"''")] 



X dR{x, z) dF{xQ, Zq) 



a—XQ<x<a 
XQ<a/2 
\zo\<\/a 



(5) 



+ o{a(-i-*)/2(i^g^yn/2| y ^ xo)dF{xo, Zo) 
[f(x~a,z) + 0{l)LUf(x-a,z;e-^'')] 



a<.x<a+XQ 
xo<a/2 



X f(x — Xq, Z — Zq) d\{x, z) dF{xo, Zq) 



as a — > oo. In the continuous case, A is a multiple of Lebesgue measure, and this 
expression can be simplified using dominated convergence. The mixed case is similar 
but more troublesome and will be treated carefully. Details for the continuous case 
are omitted. To avoid problems associated with the counting measure factor in A 
we will use the following rounding argument. Let (w) be w with the last coordinate 
rounded to the nearest integer. Then 



(6) 



where 



hdX = J h[x,T, ^^'^{w ~ ^x)) dX{x,'w) 
^jh{x,^-^'^H-l^))dxd. 
= {alv)"'''^^\ j h{x + a,ZaVa/^)dxdz 



Za — Za{x) = \fvja^ "'"/^ ((7(0 + x) + \faJvY}l'^ z) — 7(0 + x)) . 



Note that \za — z\ is bounded by some multiple of 1/ ^/a, uniformly in x. Using (p]), 
the integral in ([5]) is 



(7) 



[f{x, Za\/a/v) + 0{l)ujf{x,Za\/a/y)]Kdxdz (iF(xo,zo), 



a<x<a-\-xo 
xo<a/2 
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where 



A = 



X — Xq 



a 



m/2 



ZO 



^1 + (x - Xo)/a 

Zn 



1 + {x ~ Xo)/a 



-H 



zo 



1 + {x - xo)/a V \/l + (a; - xo)/a 1 + {x - xo)/a 

Viewing A as a function of a^^/^ (with Za treated as a separate independent vari 
able), by Taylor's theorem with Lagrange's form for the remainder, 

(8) A = Ao + 0/(2a), 

where 6 is (9^A/9^(a^^/^) evaluated at an intermediate point between and a^^/^ 



and 



1 



1 + J - {U{Za) + Zo ■ Za) 

" a 



Ao = -(piZa) 
V 

A little calculus shows that 1 6*1 is bounded by a multiple of 

(9) {1 + la^-xol +z^}(l + |z,|5)0(z,)exp{|2:,|V2^}, 

or by a multiple of this expression with z^ changed to z. By ([5]), 

a(i+*)/2(A_Ao)^0 

pointwise in (x, z, xq, zq) as a ^ oo, and by ([§]), a'^+'^-'/^IA — Ao| is bounded by a 
multiple of 

{1 + |x - a;oP+')/2 _^ (1 + |z|5)0(z) exp{|z|V2^}. 

This last expression is integrable because 

[1 + la; - xo|(i+')/2 ^ dxdF(xo, zo) 



= E 



3 + (5 ' ' 



< (X). 



Hence by dominated convergence, ([7]) equals 

[/(x, Za^/a/v) + 0{l)uj f{x^ Za\faji'\ e"''")] Aq dx dz dF{xo, zq) 



a<x<a+XQ 
xo<a/2 
No|<\/a 

+ o{a(-i-^)/2}. 

At this stage, by ^ and ^ , there is no harm removing the upper endpoints from 
the xo and zq integrations. Performing the integral against F and using 

Ef{Xr - a, Zr) 



[fix -a,z) + 0{l)ujf{x - a, a; e~"°)] 

X |/0o(a; — a) + ^/i7/a[li{z ^/uja) po{x — a) + z ■ pi{x — a)\/iy/a\ | 

(t){z^/i7/a) ~ 

X — , aXix.z) 

V^(a/j^)™/2 

+ o{a("i-*)/2(loga)™/2} 
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as a ^ c«. This proves Theorem 13. II □ 
4. Ladder variables and the proof of Theorem 11.11 

By introducing ladder variables, Theorem 11.11 can be obtained directly from the 
results of Section [3l Let Tq ^ 0, Ti = T and define later ladder times recursively 

by 

Tfc+i = inf{n > Tk : Xn > Xt^} 
for fc > 0. Also, let Xk = Xt,, Wk = Wt, and W* = W*^. Then {iXk,Wk)}k>o 
and {{Xk,W^)}k>o are random walks. Define 

f = fa = inf{fc : Xk > a}. 

Since r must be a ladder time, Xr = Xf, Wr = Wf and W* = Wf. Hence Theo- 
rem [O] follows immediately from Theorem 13. II The only work necessary is to show 
that {X,W) and {X,W*) satisfy the regularity conditions for Theorem 13.11 and 
to verify that the approximate densities in the two theorems agree. The necessary 
identities appear in Corollary [L3l and Lemma [4.41 

The first result of this section concerns moments of randomly stopped sums. Let 
ei, 62, ... be i.i.d. with Eet = and Eef — 1. Also, let the et be independently 
adapted to a filtration J^, i.e., for each n > 1, e„ is J-n measurable and independent 
of J-^n-i- Let C,n = ^^'^ i be a stopping time. 

Lemma 4.1. Let p > 0. Then for some constant K (that depends only on p), 

(10) E]^]^ <K{EtP/'^ +EtE\ei\P]. 
If Et < oo then 

(11) E[Ct-Cn;t>n]=0 
and 

(12) E[{Ct-(:nf;t>n]^E[t-n;t>n]. 
If Et^/"^ < oo and E\ei\^ < oo then 

(13) ECf ^ SEtCt + EtEel 

Proof. Equations (fTTj) and (fT2|) are Wald's first and second identities. Under more 
stringent moment conditions, (|13p was discovered by Chow, Robbins, and Te- 
icher The moment conditions are weakened in Theorem 2.1 of Brown 21, and 
([T3|) follows from (fTO|) using this result. For a narrower class of stopping times, (fTO|) 
is given by Gut [8|]. He notes (at the end of his paper) that the result can be easily 
obtained from Theorem 21.1 of Burkholder 3 (or Theorem 5.3 of Q), and that 
approach works in this setting as well. □ 

For the next lemma, let (e^^"*, e^^''), (62^^62^''), ... be i.i.d., zero mean and inde- 
pendently adapted to T. Let Cn^ — Y^=\ ^^t = 'Y^=\ eP"* and let i be a 
stopping time. 

Lemma 4.2. If E\e^^^f^'^ < 00, £'|ef^|^ < 00 and Et^^^ < 00, then 

EC^,'\l'^ ^EtEe['^e['\ 
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Proof. Since {(.n ^r?"^ — ft-£'e^^''e^^''}„>i is a zero mean martingale, it is sufficient to 
show tliat 

E[Ci'\i'' - tEe^^e^^ - Ci^^Ci^^ + nEe[%r^;t > n] ^ 0. 
This expression is the sum of two terms: 

E[n^t]t> n]Ee[^\''^^ 

and 

(14) E[Ci'\i'^-e\i'^;t>n]. 

Using Lemma [4.11 it is easy to show that £'[Ct^^ — Cn^''|-?>i] is zero on t > n. By 
Holder's inequality and (fTO| . i?|(Ct^^ — Cn^')Cn'^'l < oo, and so, conditioning on Tn, 
Eiid^'' - d'Vi'^i > n] ^ 0. Similarly, ^[(Cp^ - Cn^)d^^;t> n] = 0. Hence, (HI 
equals 

which is bounded in magnitude by 

The second factor in this last expression approaches zero by the argument for ()10|) . 
Using a square function inequality (such as Theorem 3.2 of 3]), -ElCt^"* ~ Cn^l'^^'^ is 
bounded by a multiple of 



r * , ,9i3/4 * 



E 



<Ej2\e['^f/'^EtE\e['^^'/' 



and the lemma follows. □ 

Corollary 4.3. Suppose 6 G (0,1), ly > 0, E\X\^^+^'^/'^ < oo and E\Z\^+^ < oo. 
Then i;x(3+'5)/2 < oo, ET^^+^'>/^ < oo and E\Z\^+^ < oo. Also, EX = vET, 
EW/EX = 7, Cov{W - jX) = ETT., 

(15) Ei,.zf^^^'^-'^' . 3M^i.r,.z 



/ET ET 



(16) i,z^,.z=^^+(!Ii±^l^^ 

and 

(17) E{X/v-T)Z = VeTEXZ/v. 

If the moment condition for X is strengthened to E\X\'^^^ < oo, then EX^~^^ < oo, 
ET^+^ < oo, E\Z*\^+^ < oo, EW*/EX = 7*, Cov{W* -J*X) = ETY.^ and 
identities psp and p6p relating Z to Z hold relating Z* to Z* after changing m 
to m + I. 
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Proof. By Theorem 2.1 of Gut 8], if p > 1, EX > and E\X\p < oo, then 
ETP < oo and EXP < oo. So ((171) follows from Lemma [42] with (n^ = X„/t/ - n 

(2) 

and Q an arbitrary coordinate of Z„. The other assertions follow easily from 
Lemma [4. II For instance, by (I12p with n — 0, 

Var(g • {W - jX)) = £;(g • (Wr - T^t))^ 

= ETYnT^q- {W --fX)). 

So g • Cov(TI^ - jX)q = STg • Sg, which imphes Cov(W" - jX) ^ ET S. To obtain 
dUl), let i aV^gf act on □ 



Lemma 4.4. Suppose EX > and (X, W) satisfies Cramer's condition. Then T 
is arithmetic on Z and {X, W) is strongly nonlattice with T . 

Proof. Introduce T- = inf{n : X„ < 0}. In the notation of Greenwood and 
Shaked 0], T and T_ are dual stopping times, and by their multivariate exten- 
sion of the Wiener-Hopf factorization theorem, 

1 _ £;gi(«iX+52-W+C3) 

= |l - £:e^(«i'*+«2-*'+«3T)| |i _ ^ g*(6XT_+«2-WT_+«3T_).y_ < OO I . 

The lemma now follows easily since P(T_ < oo) < 1. □ 



5. Marginal distributions 

To organize the proof of Theorem II. 2i it is convenient to approximate several 
marginal distributions separately. Lemma 15.11 gives marginal distributions for WV 
and W* . Corollarv l5.2l gives the marginal distribution for the first component of Wr , 
which establishes Theorem 11.21 when h is linear. Lemma 12.21 shows that variables 
sufficiently close in a natural sense have the same asymptotic expansion. Lemma [5.3l 
provides an expansion for a special class of variables close to the variables in The- 
orem [L2l 

Let Ao be Lebesgue measure on R™ and be the product of Ao with counting 
measure on Z. Define Qo and Qq by 

dQo, ^ (t>[q) . rTn 

= ^T^f7773;;:7jU + V^/aHo(g)| 

and 



dXo y|E|(a/z/)"/2 



where 



—rrri^ ) = — 7= : z-r \ 1 + V ^^/fl-Hnlg ) \, 

dXy ' 

q = ^^^^^{w — 7a)-\/ v/a, 
q* = S* ^^^{w* — ^* a)\J V I a, 
Ho{q)^^E{q-Zf-^EZ^-Z 

-I- — ^-^EXq ■ Z + -EX^ 

2v ^ 2vET 
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and 



H;5(g*) = ^E{q* ■ Z*f - ^EZ*^q* ■ Z* 



Lemma 5.1. Suppose {X^W) satisfies Cramer's condition, v > 0, E\X\'^^^ < oo 
and E\Z\'^^^ < oo, where S G (0, 1). Then for some rj > 0, 

(18) Ef{Wr) = f fdQo + 0(1) / u;/(- ; e""") dQo + o{a^-^-'^^^ (log ar^^} 



as a oo, uniformly for nonnegative measurable f hounded above by one. If the 
moment condition for X is strengthened to E\X\'^^^ , then for some r] > 0, 

(19) Ef{W:)^ ffdQ*+0{l) fujf{-;e-'ndQ*o+o{a^-'-'^/\\oga)^"^+'y'} 



as a oo, uniformly for nonnegative measurable f bounded above by one. 
Proof. Using Theorem I l.l) the only difficult task verifying (fT8| is to show that 

(20) / f{w)dQ{x,w) - / fiw)dQo{w)+o{a^-'~'^/\\ogar/'}. 



Integrating over w, the Q marginal density for x is bounded by some multiple of 
Poix) + \pi{x)\/^/a. Since 

Po{x)dx^^E{X 

< E\X^+^-X>^ 

- vET ^ ' ^ ^ 

(-l-A-)/21 



and 



^/ \pi{x)\dx<^=E[\Z\X-X>^ 



{-l-S)/2 

< , E\\Z\X^+'-X > V^l 

Q{[\/a, oo) X M™} — o{a^~^"'^''/^}. Hence there is no harm restricting the domain 
of integration in (|20p to a; < \/a. Now q — q = —T,~^^^"/Xy/ v j a, so any intermediate 
point on the line segment from q to q equals 

q-QYT^I'^^x^JVjTL 

for some 6 E [0, 1]. The squared length of this vector is 

\q\^ - 2eq ■ Yr^^'^-iXyJ17[ix + B^x^^i ■ T,-'^-/iy/a > \q\^ - 2\q ■ Yr^l'^-ixyJVll\. 



74 



R. Keener 



Using Taylor's theorem with Lagrange's form for the remainder, 
and 

10(9) - m - q ■ 2-1/27x7^7^0(9)1 



1 

< - 
- 2 



Also, 
and 



X —(j)(q) exp |g • YT^I'^^xJ v j a\. 
a 

\q ■ pi{x) - q ■ pi{x)\ < \pi{x)\ \Yr^l'^-i\x^JVi7i, 



7 • E 



|H(g)-H(g)| < A'(|g|-Vl)x/V^, 
for some X > on a; < ^/a. Using these bounds, for an appropriate constant if, 



dX 



{x,w) 



m 



-— j|po(a;) + \/VJa\¥i.{q)pa{x) -^q- pi{x) + q ■ Y. '^^'^-yxpoix 



+ (1 + |,r)(.i+Vo(.) +:.^|pi(x)|)^^^^^o{a(-i-^)/2} 

as a — > 00, pointwise in q and x. With o changed to O, this result holds as a — > 00, 
uniformly for w € R™ and x < ^/a. Hence, by dominated convergence 



fdQ 



po{x) + y^ly/a[ll{q)pQ{x) + q ■ pi{x) + q ■ T, 1/272:^0(2;)] } 
a\{x,w) 



V^(a/z/)™/2 
+ o{a(-i-*)/2} 



v^ET 
f(w)(j)(q) , , 



□ 



'lS|(a/z/)™/2 

+ o{a(-i-*)/2} 

as a — > 00. This gives ([T5|. The proof of (fTO|) is similar 

When m — 1, integration of dQo / dXo gives the following corollary. 

Corollary 5.2. If m ^ 1, {X,W) satisfies Cramer's condition, v > 0, £'|Xp+* < 
00 and \Z\'^^^ < 00, where S € (0, 1), then 

P{Wr < w) = ^(w') + \/^4>{w)lii{w) + o{a(-i-'')/Vloga} 

as a 00, uniformly inw, where li — (w — ja) jicr^J ajv), = Yi = Var(Ty — 7X), 
and 



lli{w)^{w' -1){--EZ 



1 



EXZ^ -fEX^ 
2v J ^ 2U^Ef' 
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For the next result, let y„ denote the first coordinate of Wn, so W„ = (F„, Vn) 
where K € M""!. Also, let V* = (K, «), so W* = n) = (F„, V*). Partition 

and 7* as 

^.= (y'' f') and j*=(^' 

\^21 ^22 / V72 

where Sn is 1 by 1, S22 is to by m, 71 e M and 72 G K™. Let 

5^11-2 = Sii — I]i2S22^5]21 

and 

£22-1 = S22 — I]2lSj^j"'"S]i2. 

The two factorizations of the multivariate normal density as a marginal times a 
conditional density give 

Hq*) Hv*) <t>{yo-nv*) 



(21) V^(a/^.)(™+i)/2 ^E^(a/z.)W2 ^E^^^ 

^ '/'(y) 0(^0 - r2y) 

where = (y, w), 

_ „-l/2„ ,p-l/2 _ ^-l/2„ ^-1/2 

ri — ^11.2 ^12^22 ' '^2 — ^22-1 ^21^11 

2/0 = Sj^i^f (2/ - lia)^JVj7i, = S22a^(«* - 72a) V^Va 

and 

y = ^^(y - 7ia)yiVa, = ^^-i^'^i^* " l2a)\fvja. 

Let = (Ft- — 7ia)/-/a and V* ~ iy* — ^2a)/\/a. Finally, let Ai denote the 
product of Lebesgue measure on M™^^ with counting measure on Z. 

Lemma 5.3. Let ho G K, hi : M'" ^ R a homogeneous linear function and h2 : 
M™ ^ R a homogeneous quadratic function given by h2{q) = q ■ Aq for some 
symmetric m by m matrix A. Assume {X, W) satisfies Cramer's condition, v > 0, 
E\X\^+^ < 00 and E\Z\^+^ < 00, where S G (0, 1). Define 

„ _ Y*+ [Y*hi{V*) + h2{V*)]/VE 
^° ~ 1 - hoY*/^ 

Then 

P(So < c) = Fa{c) + o{a(-i-*)/2(loga)('"+i)/2} 

as a —^ 00 uniformly in c, where Fa is the approximate distribution function in 
Theorem 

Proof. By Theorem ll.il for any e > 0, 

P{\Y*\ > eV^) = o{a(-i-*)/2(i^g^)(m+i)/2| 

and 

Pi\V*\ > eVH) = o{a(-i-^)/2(loga)(™+i)/2} 

as a — > cx). Hence 

- l + [c/io + /^i(V^#)]/y^r ^ ^ ^ ^ ^ 



p(So<c) = p r 
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as a — > CX3, uniformly if jcj/y^ stays sufficiently small. Using Lemma lS.li this prob- 
ability to the order of accuracy desired can be obtained by integrating the density 
of Qq over the appropriate set. Using the first factorization of pi]) , integration over 
y gives 



(22) 



2 



X |$(c^,.) + y^[po(w*)«'(c„.) + piiv* , Cy^)(l){cy*)]^ dXi{v*), 



where po and pi are polynomials (their exact form is not important) and 



l+[cho + hi{^li^i*/^)]/^ 
As a — *■ oo, 

Cv' = Cy. - x{v*)/Vo' + 0{l/a) 
pointwise in v* and c, where 

and 
Then 

$(c„0 = $(c..) - (l>{cy*)x{v*)/V^+0{l/a), 
<i>{cy,)/^=<Picy,)/^+0{l/a) 

and 

Pi(?;*,c„.)0(ci,.)/Va = pi{v* , Cy.)(l){cy*)/^/a + 0{l/a) 

as a ~> oo, pointwise in and c. For some e > 0, with the error rate changed to 
oja^^^^*^/^}, these three relations will hold uniformly for < a' and |c| < a'^. 
This is sufficient to justify the obvious substitutions into ((22|) . This has a net effect 
of changing Cy* in ()22|) to c^,. and adding an additional term: 



(23) -[^m^^dx.i.n. 

Without this extra term, the integral is simply the approximation for P{W'^ < c). 
To finish it is sufficient to show that ((23|l equals 



fV(a^ + + ^Er^^E^r + ^tr[AE,,] 1 + o{a(--)/^}, 



as this is the difference between Fa{c) and the approximation for P{W'^ < c) 
obtained from Corollary [521 Using both factorizations in ((2T|). 



0(?;*)0(ct,.) (/)(c) 0(w5-r2c) 



■v/Sii.2a/i 
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In the integral in ((23|) there is no harm changing the measure of integration from Ai 
to Lebesgue measure. This can be verified using the Euler-MacLaurin sum formula: 
If I /'I is integrable, then 



E /(") = / fi^)dx + J {x)ofix)dx 



where is the periodic function defined for x e [0, 1) by (x)o — x ~ 1/2. In ([23|) . 
partial derivatives with respect to the last component of v* introduce an extra factor 
of i/a in the denominator, so changing Ai to Lebesgue measure changes the integral 
by 0{l/a). Since 4i{vq — r2c)/{^/\'E22■l\ia/h')"^^'^) is a multivariate normal density, 
([23]) has been simplified to the expectation of a quadratic function of a normal 
vector. The lemma then follows from standard facts about the normal distribution. 
Uniformity for all c follows from uniformity for |c| < a*^ by monotonicity of the 
distribution function being approximated. □ 

Proof of Theorem \1.'A Using Lemma 12.21 this proof will be accomplished showing 
that 

a(i+'5)/2 



Vloga 

in probability at rate o{a^~^~'^)/^\/Ioga} . Using Theorem 4 of Lorden fl^ , for some 
if > 0, 



(24) 



as a — > oo, since the conditions of Theorem 11.21 ensure 

(25) a(i-*)/2(X,-a) ^0 

in probability at rate oja'^"'^^*'/^}. Applying Theorem ll.il with the random walk 
changed to {{Xn, Yn)}, for some constant K > 0, 

P{\Y*\ > K^/l^) = o{a^-^-^^/^^/k^}. 

Similarly, applying Theorem ll.ll to two dimensional random walks where the second 
coordinates are projections of Wn, for some K > 0, 

(26) P{\W*\ > i^yioia) = o{a^-'^-^^^^^/k^}. 
Let 

Si = y# + [hoY*^ + Y*hi{V*) + h2{V*)\ /v^. 
By Taylor expansion, on {|W^*| < K-J\oga], 

So-Si =0{(loga)3/2/a}. 

Consequently, 

„(i+^)/2(s„_5^)^0 
in probability at rate o{^a'--~^^^^/^-J\oga] . Next, let 

S2 = Si + h^iXja - If + {Xja - l)K{Y,/a - -fuV:/a - 72), 
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the 2-term Taylor approximation for ^. Using (|25p and ((26)) . 

in probability at rate o{a'-^^^^^^^y/loga} . By ([24]) and ([26l) . 

P(5;/a ^ No) = o{a(-i-*)/2v/l^} 

as a ^ 00, for any neighborhood A'o of sq. With Nq sufficiently small and K large 
enough, on {S*/aeNo] 

S-S2 <i^V^{|X,/a-l|3 + |iy;/a-7|3} 

by Taylor's theorem. Using ([^5]) and it follows that 

(S-S2)a(i+^'/Vv^->0 

in probability at rate o^a'^^^^^^'^^y/loga} , which proves Theorem 11.21 □ 
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